It is often desirable to configure a website to best meet the needs of visitors of the website. Many visitors, however, are not interested in taking the time to provide feedback concerning a website. Accordingly, experimentation tools have been developed to test the popularity of features, to test how visitors react to certain features, and to otherwise track visitor interaction with a website in order to obtain feedback which may be used to modify the website to improve the service provided to visitors.
A situation that may arise in the context of website experimentation is that a given website may be constantly changing. Features available on the website at one point in time may not be available at another point in time. Such changes may result from, for example, service interruptions or failures, changes in the program logic that supports hosting of the website, the operation of system optimization algorithms, changes in advertising campaigns, and/or other random or pseudo-random disturbances in the operation of the website. A website that is constantly changing is often seen as posing a problem for website experiments. That is, to the extent that such changes are constantly occurring in the way the website is presented to visitors, it is considered more difficult to conduct carefully planned tests over an extended period of time.
An ongoing need exists for systems and methods that may be used to perform website experimentation. It will be appreciated that while the teachings herein describe certain features and advantages that may be achieved, the teachings herein may be used to implement systems and methods that do not necessarily achieve any of these features or advantages, but rather achieve other features and advantages.